Techniques for Thermal Modeling of Data Centers to Improve Energy Efficiency

ABSTRACT

Techniques for modeling a data center are provided. In one aspect, a method for modeling a data center is provided. The method comprises the following steps. Spatially dense three-dimensional thermal distribution and air flow measurements made in the data center using a mobile off-line surveying system are obtained. A temperature and air flow model for the data center is created using the spatially dense three-dimensional thermal distribution and air flow measurements. The temperature and air flow model is used to make thermal distribution and air flow predictions of the data center. The thermal distribution and air flow predictions are compared with the thermal distribution and air flow measurements made using the mobile off-line surveying system to produce a validated model for the data center.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is related to the commonly owned U.S. application Ser.No. ______, entitled “Techniques to Predict Three-Dimensional ThermalDistributions in Real-Time,” designated as Attorney Reference No.YOR920080115US1, filed herewith on the same day of ______, the contentsof which are incorporated by reference herein.

FIELD OF THE INVENTION

The present invention relates to data centers, and more particularly, totechniques for modeling data centers.

BACKGROUND OF THE INVENTION

The heat dissipated by today's computing equipment is reaching levelsthat make it very challenging to cool these systems in densely packeddata centers or telecommunications rooms. In data centers, the computingequipment, such as a multitude of computer servers, are commonly placedin a series of racks arranged in a series of aisles in the data center.Typically, a data center has a cooling system that, e.g., by way of oneor more air conditioning units (ACUs), introduces cooled air to theracks, for example, through a sub-floor plenum and associated perforatedtiles in the floor above the sub-floor plenum. Without a proper layoutin the data center, costly inefficiencies in the cooling systeminevitably occur, and can potentially result in ineffective cooling ofthe equipment.

Air flow distributions within a data center have a major impact on thethermal environment of the equipment within the data center.Computational fluid dynamics (CFD) calculations have been used to solvethe Navier Stokes (NS) Equations and the modeling results of the NS-CFDhave been employed to thermally manage data centers. There can be,however, several potential problems associated with NS-CFD modeling of adata center. First, while NS-CFD modeling has been successfully deployedfor the design of very well-defined structures, such as air plane wings,the application of NS-CFD modeling to data centers can be somewhatproblematic because input data needed for NS-CFD modeling is often notavailable and/or is inaccurate. Namely, every data center is differentand a current inventory list is often not available for each data center(further, heterogeneous technology may be used within a given datacenter, e.g., computer equipment from different vendors and/or ofdifferent vintages), available data (e.g., name-plate power and flowdata) generally does not reflect actual usage, air flow is verydifficult and time-consuming to accurately measure and characterize (andoften does not capture room effects such as drafts). In all, it couldeasily take one person at least one week to survey a 5,000 square footdata center, which is an overly time-consuming process.

Second, a data center NS-CFD model can be difficult to generate andtypically requires a detailed survey of the data center, which is a timeconsuming and costly process (as described above). However, even if acomplicated NS-CFD model has been built, there is very little confidencethat it actually gives dependable insights, because, as described above,the input data for the NS-CFD model is often not available and/or doesnot accurately represent the data center.

Third, the calculations involved are time-consuming (slow) and besidesmany assumptions, which are intrinsically built into these NS-CFD models(i.e., with CFD and other related models such as the k-epsilonturbulence model, uniform air flow rate through server rack and uniformvolumetric heat generation inside of the server rack are assumed) suchmodels cannot readily include spatial and temporal variability in theworkload, as well as other unknowns, because the pure computation timeof these models is quite significant.

Fourth, existing data center NS-CFD models cannot easily be used tooptimize data center layout. Namely, there is no systematic strategy forchanging inputs to the model based on measurements. Rather, data centeroptimization is done today rather unscientifically “by hand” playing afew what-ifs, e.g., where an engineer looks at results and uses his/herintuition to adjust the model.

Therefore, data center modeling techniques are needed that provideimproved accuracy and efficiency over conventional processes so as topermit optimization of data center layout.

SUMMARY OF THE INVENTION

The present invention provides techniques for modeling a data center. Inone aspect of the invention, a method for modeling a data center isprovided. The method comprises the following steps. Spatially densethree-dimensional thermal distribution and air flow measurements made inthe data center using a mobile off-line surveying system are obtained. Atemperature and air flow model for the data center is created using thespatially dense three-dimensional thermal distribution and air flowmeasurements. The temperature and air flow model is used to make thermaldistribution and air flow predictions of the data center. The thermaldistribution and air flow predictions are compared with the thermaldistribution and air flow measurements made using the mobile off-linesurveying system to produce a validated model for the data center.

It can then be determined whether a margin of error between the thermaldistribution and air flow predictions and the thermal distribution andair flow measurements is acceptable. One or more changes can be made tothe temperature and air flow model if the margin of error between thethermal distribution and air flow predictions and the thermaldistribution and air flow measurements is unacceptable. The use, compareand determine steps can be repeated until the margin of error betweenthe thermal distribution and air flow predictions and the thermaldistribution and air flow measurements is acceptable.

A more complete understanding of the present invention, as well asfurther features and advantages of the present invention, will beobtained by reference to the following detailed description anddrawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating an exemplary data center according toan embodiment of the present invention;

FIG. 2 is a diagram illustrating an exemplary methodology for creating atemperature/air flow model for a data center according to an embodimentof the present invention;

FIG. 3 is a diagram illustrating an exemplary methodology for validatingand using the present temperature/air flow models according to anembodiment of the present invention;

FIG. 4 is a diagram illustrating air temperature contours and flowprofiles resulting from use of a temperature and air flow model on anexemplary data center according to an embodiment of the presentinvention

FIG. 5 is a diagram illustrating air flow potential and field in anexemplary data center according to an embodiment of the presentinvention;

FIG. 6 is a diagram illustrating temperature field in an exemplary datacenter according to an embodiment of the present invention;

FIG. 7 is a diagram illustrating an exemplary data center having asub-floor plenum according to an embodiment of the present invention;

FIG. 8 is a diagram illustrating an exemplary apparatus for modeling adata center according to an embodiment of the present invention;

FIG. 9 is a diagram illustrating an exemplary methodology for creating atemperature flow model using mobile measurement technology according toan embodiment of the present invention;

FIGS. 10A-C are plan view temperature contour plots illustrating acomparison of experimental and data center model data with anunacceptable amount of error present according to an embodiment of thepresent invention; and

FIGS. 11A-C are plan view temperature contour plots illustrating acomparison of experimental and data center model data with an acceptableamount of error present according to an embodiment of the presentinvention.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

FIG. 1 is a diagram illustrating exemplary data center 100. Data center100 has information technology (IT) racks 101 and a raised-floor coolingsystem with air conditioning units (ACUs) 102 (also referred to hereinas computer room air conditioners (CRACs), see below) that take hot airin (typically from above through one or more air returns in the CRACs)and exhaust cooled air into a sub-floor plenum below. Hot air flowthrough data center 100 is indicated by light arrows 110 and cooled airflow through data center 100 is indicated by dark arrows 112.

In FIG. 1, IT racks 101 use front-to-back cooling and are located onraised-floor 106 with sub-floor 104 beneath. Namely, according to thisscheme, cooled air is drawn in through a front (inlet) of each rack andwarm air is exhausted out from a rear (outlet) of each rack. The cooledair drawn into the front of the rack is supplied to air inlets of eachIT equipment component (servers for example) therein. Space betweenraised floor 106 and sub-floor 104 defines the sub-floor plenum 108. Thesub-floor plenum 108 serves as a conduit to transport, e.g., cooled airfrom the ACUs 102 to the racks. In a properly-organized data center(such as data center 100), racks 101 are arranged in a hot aisle—coldaisle configuration, i.e., having air inlets and exhaust outlets inalternating directions. Namely, cooled air is blown through perforatedfloor tiles 114 in raised-floor 106, from the sub-floor plenum 108 intothe cold aisles. The cooled air is then drawn into racks 101, via theair inlets, on an air inlet side of the racks and dumped, via theexhaust outlets, on an exhaust outlet side of the racks and into the hotaisles.

The ACUs typically receive chilled water from a refrigeration chillerplant (not shown). Each ACU typically comprises a blower motor tocirculate air through the ACU and to blow cooled air, e.g., into thesub-floor plenum. As such, in most data centers, the ACUs are simpleheat exchangers mainly consuming power needed to blow the cooled airinto the sub-floor plenum.

Typically, one or more power distribution units (PDUs) (not shown) arepresent that distribute power to the IT equipment racks 101. As will bedescribed in detail below, power consumption by the PDUs can be animportant consideration in the present techniques. In general, since thePDUs supply electrical power required by the IT equipment in a datacenter, a total electrical power intake of the PDUs represents animportant parameter in determining the energy efficiency of a datacenter.

FIG. 2 is a diagram illustrating exemplary methodology 200 for creatinga temperature and air flow model for a data center. In step 202,spatially dense three-dimensional thermal distribution and air flowmeasurements are made in the data center, which will serve as input datafor creating the model. According to an exemplary embodiment, thethree-dimensional thermal distribution and air flow measurements aremade using a mobile off-line surveying system (MOSS), such as mobilemeasurement technology (MMT) V1.0 (for example, MMT V1.0 temperaturemeasurements are spatially dense, e.g., typically spaced about eightinches apart from one another in x, y lateral dimensions and about 12inches apart from one another in z dimensions). MMT V1.0 is described inU.S. Pat. No. 7,366,632, issued to Hamann et al., entitled “Method andApparatus for Three-Dimensional Measurements” (hereinafter “U.S. Pat.No. 7,366,632”) the contents of which are incorporated by referenceherein. MMT V1.0 is a technology for optimizing data centerinfrastructures for improved energy and space efficiency which involvesa combination of advanced metrology techniques for rapidmeasuring/surveying data centers (see, for example, U.S. Pat. No.7,366,632) and metrics-based assessments and data-based best practicesimplementation for optimizing a data center within a given thermalenvelope for optimum space and most-efficient energy utilization (see,for example, U.S. application Ser. No. 11/750,325, filed by Claassen etal., entitled “Techniques for Analyzing Data Center Energy UtilizationPractices,” designated as Attorney Reference Number YOR920070242US1, thecontents of which are incorporated by reference herein).

The measurements gathered from MMT V1.0 are an excellent starting pointfor building a data center model. The MMT V1.0 measurements not onlyprovide necessary physical parameters for the model input but also allowfor direct validation and correction (e.g., if the initial modelpredictions disagree with measured temperature/air flow measurements, asdescribed in detail below). For example, an actual data center floorplan (many data center managers do not have a current floor plan) can bereadily deduced from the MMT V1.0 measurements as MMT V1.0 is equippedwith a positioning tracking system, see U.S. Pat. No. 7,366,632.

As highlighted above, some conventional approaches involve models thatuse inputs, such as air flow and power. With one common approach, datacenter thermal and air flow modeling involves solving Navier-Stokes (NS)equations for irrotational flow (∇×{right arrow over (v)}=0) usingintensive computational fluid dynamics calculations, which can compriseas many as five coupled (non-linear) partial differential equations (onefor conservation of mass, three for momentum conservation and one forconservation of energy). Using a description of the fluid or gas (suchas the ideal gas law which relates pressure p and temperature T) thereare six equations for six unknowns (p, v_(x), v_(y), v_(z), T, ρ) withfour variables x,y,z,t (see below). In order to solve these equations,Navier-Stokes-computational fluid dynamics (NS-CFD) simulations aretypically used, i.e., CFD is used to obtain numerical solutions.

The NS computations can be carried out as follows:

$\mspace{20mu} {{{\frac{\partial\rho}{\partial t} + {{div}\left( {\rho \; \underset{\_}{v}} \right)}} = 0},\mspace{79mu} {{\frac{\partial\underset{\_}{v}}{\partial t} + {\underset{\_}{v}\; {{div}\left( {\rho \; \underset{\_}{v}} \right)}}} = {{{- \frac{1}{\rho}}{{grad}(p)}} + {\frac{v}{\rho}{{grad}\left( {{div}\left( \underset{\_}{v} \right)} \right)}} + \underset{\_}{F}}},{{{\rho \; c_{p}{\underset{\_}{v}\begin{pmatrix}{\frac{\partial T}{\partial t} +} \\{{grad}(T)}\end{pmatrix}}} + {{div}\left( {k\; {{grad}(T)}} \right)} + {p\; {{div}(v)}} + h + {\upsilon \; \Phi} + {\rho \; c_{p}\frac{\partial p}{\partial t}}} = 0},\mspace{79mu} {{\rho \left( {P,T} \right)} = \frac{Mp}{RT}},{{\nabla{\times \overset{->}{v}}} = 0},}$

wherein C_(p) is specific heat, ρ is air density, v is velocity vector,υ is viscosity, F is external force, M is molar mass, Φ is viscousdissipation function, h is power dissipation/heat removal, T is absolutetemperature, t is time, p is absolute pressure of medium (e.g., air) andR is the universal gas constant. According to one exemplary embodiment,in step 204, the MOSS, e.g., MMT V1.0, three-dimensional thermaldistribution and air flow measurements are used in the NS-CFDcalculations to provide a temperature and air flow model for the datacenter. The creation of a NS-CFD model using MMT V1.0 measurements isdescribed, for example, in conjunction with the description of FIG. 9,below, and in Y. Amemiya et al., “Comparison of Experimental TemperatureResults with Numerical Modeling Predictions of a Real-World Compact DataCenter Facility,” Proceedings of IPACK2007 ASME InterPACK '07 (July2007) (hereinafter “Amemiya”), the contents of which are incorporated byreference herein.

However, by way of the present teachings it has been discovered thatsince the thermal distribution and air flow measurements taken using,for example, MMT V1.0 are spatially dense (as described above), certainassumptions can be made about conditions in the data center. Accordingto an exemplary embodiment, it is assumed that there is an irrotationalvelocity field, no turbulence, constant density, free slipping overboundaries (slipless) and viscous forces are much less than inertialforces (i.e., viscous forces can be neglected). By way of theseassumptions, the Navier-Stokes equations (shown above) can be simplifiedas follows (wherein a slash indicates the cancellation of a term),

$\left. \mspace{79mu} {{{+ {{div}\left( {\rho \; \underset{\_}{v}} \right)}} = 0}\mspace{79mu} {{{+ {\underset{\_}{v}{{div}\left( {\rho \; \underset{\_}{v}} \right)}}} = {{{- \frac{1}{\rho}}(p)} + {\frac{v}{\rho}\left( {{div}\left( \underset{\_}{v} \right)} \right)} +}},{{\rho \; c_{p}\underset{\_}{v}} + {{grad}(T)}}}} \right) + {{div}\left( {{{{k\mspace{14mu} {{grad}(T)}} + {p\mspace{14mu} {{div}(v)}} + h + +} = 0},\begin{matrix}{\mspace{79mu} {{\rho \left( {P,T} \right)} =}} & {{\nabla{\times v}} - 0.}\end{matrix}} \right.}$

As such, terms

$\frac{\partial p}{\partial t},\frac{\partial\underset{\_}{v}}{\partial t},{{- \frac{1}{\rho}}{{grad}(p)}},{\frac{v}{\rho}{grad}},\underset{\_}{F},\frac{\partial T}{\partial t},{\upsilon\Phi},{\rho \; c_{p}\frac{\partial p}{\partial t}\mspace{14mu} {and}\mspace{14mu} \frac{Mp}{RT}}$

can be cancelled.

With zero divergence (volume conservation), i.e., ∇v=0, and zerorotation, i.e., v=∇φ, the above simplified equations provide atemperature and air flow model for the data center, i.e.,

∇²φ=0  (1)

ρc _(p) vgrad(T)+div(k grad(T))+h=0,  (2)

wherein ∇²φ=0 (i.e.,

${{\nabla^{2}\varphi} = {{\frac{\partial^{2}\varphi}{\partial x^{2}} + \frac{\partial^{2}\varphi}{\partial y^{2}} + \frac{\partial^{2}\varphi}{\partial z^{2}}} = 0}},$

see description of potential flow theory, below) is an air flow fieldand ρc_(p) vgrad(T)+div(k grad(T))+h=0 is a temperature field (wherein ρis air density, c_(p) is specific heat, v is velocity vector, T isabsolute temperature, h is power dissipation/heat removal and k isthermal conductivity). Thus, according to another exemplary embodiment,in step 206, MOSS, e.g., MMT V1.0, measurements are used to create amodel based on Equations 1 and 2 (see description below). WithinEquation 2 (temperature equation), the term ρc_(p) v grad(T) representsconvection, the term div(k grad(T)) represents conduction and the termh, i.e., power dissipation/heat removal, collectively represents heatgeneration. This new model is referred to herein as a “Laplacian model”because it uses a Laplace-type of equations. As highlighted above, inthe Laplacian model air flow modeling is separated from temperaturemodeling which keeps the Laplacian model linear (non-coupled) (meaningthat two solutions can be superimposed to yield a third solution). Aswill be described in detail below, the Laplacian model can be coupleddirectly to the MMT V1.0 measurements yielding a benchmarked data centermodel. In fact, the Laplacian model can be completely based on measuredinput data, which is readily and rapidly available through MMT V1.0.

Specifically, potential flow theory is employed assuming constant(temperature independent) air density, free slipping over boundaries andthat viscous forces can be neglected, i.e.,

${{\nabla^{2}\varphi} = {{\frac{\partial^{2}\varphi}{\partial x^{2}} + \frac{\partial^{2}\varphi}{\partial y^{2}} + \frac{\partial^{2}\varphi}{\partial z^{2}}} = 0}},{and}$${v_{x} = \frac{\partial\varphi}{\partial x}},{v_{y} = \frac{\partial\varphi}{\partial y}},{v_{z} = \frac{\partial\varphi}{\partial z}},$

wherein φ is flow potential and v_(x), v_(y) and v_(z) are air flowvelocity components in x, y and z directions, respectively. Equation 1(air flow equation) above cannot be solved without applying the correctboundaries. In the present teachings, e.g., MMT V1.0, supplies theseboundaries as measurements which advantageously can be fed directly intothe model. For example, the perforated tiles (or output of the CRACs)are sources (e.g.,

$\frac{\partial\varphi}{\partial z} = {- ({measured})}$

output air flow from a perforated tile) and the air returns to the CRACscan be sinks (e.g.

$\frac{\partial\varphi}{\partial z} = {+ ({measured})}$

CRAC air flows), while the racks are sinks (e.g.,

$\frac{\partial\varphi}{\partial z} = {+ ({measured})}$

inlet rack air flow) at the inlet(s) and sources (e.g.

$\frac{\partial\varphi}{\partial z} = {- ({measured})}$

outlet rack air flow) at the outlet(s).

In order to solve Equation 1, an actual relative value for the flowpotential φ=0 has to be set somewhere in the data center. In some cases,it may be desirable to set the flow potential to zero at the air returnsof the CRACs (i.e., assuming an infinite sink) or in an area where itcan accurately be assumed that that there is minimal air velocity, suchas behind one of the CRACs. All of the source and sink flow boundariescan be obtained directly from the MMT V1.0 measurements, and thus theair flow field as described by Equation 1 can be solved. Specifically,the MMT V1.0 measurements define “flow boundaries” of Equation 1. Outerparts of the modeling domain, such as the data center walls and ceiling,have natural boundaries as is commonly applied in partial differentialequation problems.

In general, with any of the data center models described herein, theinputs can include, but are not limited to, the perforated tile air flowrates and temperature, the rack heat loads (i.e., power dissipated bythe IT racks, which is equal to the power consumed by the racks toperform computational work) and (inlet/outlet) air flow rates, the CRACsair flow rates (which is equivalent to supplied cool air flow rate orreturn hot air rate), as well as several other miscellaneous parameters.These miscellaneous parameters can include, but are not limited to, tileperimeter leakage flow rates, location of cable openings (which canserve as air passages, e.g., into/out of the racks/the data center)and/or direction of CRAC(s) air flow. See, for example Amemiya.

Although neglected in this implementation, a temperature dependence ofthe air density can be included by taking the MMT V1.0 thermalmeasurements into account and by subsequently superimposing a densitydriven air velocity (Bernoulli's law). Another more simplified approachwould be to simply superimpose an up-drift velocity (in a z-direction)for a temperature difference between a bottom and a top of the datacenter.

An actual thermal distribution T (x, y, z) can then be calculated basedon the assumption that all heat transport within the data center isgoverned simply by mass transport (diffusivity α=0). However, inprinciple, thermal conduction through air can be included in thecalculations. By excluding the power dissipation/heat removal term h inEquation 2 (the temperature equation) above, Equation 2 can bere-written as follows:

$\begin{matrix}{{{\frac{\partial}{\partial x}\left\lbrack {{\alpha \frac{\partial T}{\partial x}} - {v_{x}T}} \right\rbrack} + {\frac{\partial}{\partial y}\left\lbrack {{\alpha \frac{\partial T}{\partial y}} - {v_{y}T}} \right\rbrack} + {\frac{\partial}{\partial z}\left\lbrack {{\alpha \frac{\partial T}{\partial z}} - {v_{z}T}} \right\rbrack}} = 0.} & (3)\end{matrix}$

Again, Equation 3 cannot be solved without applying the appropriateboundaries, which are directly available from the MMT V1.0 measurements(which measures the inlet and outlet temperatures for each rack). Theseinlet and outlet temperatures can be directly applied as boundaries toEquation 3, and thus the remaining temperature field can be solved.Alternatively, in combination with the flow boundaries across each rackand CRAC, an actual heat dissipation can be calculated (i.e.,h=ρc_(p)v(T_(out)−T_(in))) and Equation 2 can be used to calculate thetemperature field. As highlighted above, it can be assumed that the airdensity, as well as the specific heat, are temperature independent.However, because the thermal distribution is known from the MMT V1.0measurements, this can be corrected, for example, by taking adifference, i.e., define a spatial offset in temperature, between thecalculated temperature field and the measured temperature field (MMTV1.0). This difference (T is offset) can then be applied for predictionusing the Laplacian model. The actual equations are solved usingstandard partial differential equation solvers.

Thus, as described above, the actual MMT V1.0 measurements can be usedto set boundaries in the present data center models. Because theLaplacian model is based solely on experimental data, the Laplacianmodel represents critical temperatures (i.e., at the inlets and outlets)in the data center accurately. Simple what-ifs can then be tried outwith the model. For example, a new temperature and air flow distributioncan be calculated for different air flow rates and/or different powerconsumptions within the racks.

As highlighted above, the Laplacian model predicts the criticaltemperatures (i.e., at the inlets and outlets) correctly. The remainderof the predicted thermal distribution can then be easily validated withthe MMT V1.0 measurements, as described below. FIG. 3 is a diagramillustrating exemplary methodology 300 for validating and using thepresent temperature and air flow models. As described in conjunctionwith the description of methodology 200 of FIG. 2, above, thetemperature and air flow model(s) created using the MOSS, e.g., MMTV1.0, measurements can be a NS-CFD model and/or a Laplacian model. Instep 302, the model (NS-CFD or Laplacian) is used to predict a thermaldistribution and air flow in the data center. In step 304, the thermaldistribution and air flow predictions from the model are compared to thecorresponding data collected using MMT V1.0, which according to anexemplary embodiment includes calculating error via simple subtractionand/or by means of any suitable statistical metric(s) to quantify theaccuracy of the model. Namely, if it is determined that the thermaldistribution and air flow values predicted by the model align with theMMT V1.0 measurements to within an acceptable margin of error, then instep 306 the model is considered validated. On the other hand, if it isdetermined that a margin of error between the thermal distribution andair flow values predicted by the model and the MMT V1.0 measurements isunacceptable, then in step 308 one or more changes are made to themodel. For example, if the calculated temperature distribution does notagree with predicted temperature distribution, a three-dimensional arraywith difference (an error correction array) can be calculated. Thisarray can then be used to correct future predictions. If desired, theerror correction array can be parameterized.

The model predictions are then again compared with the MMT V1.0measurements. These steps can be repeated until a validated model isachieved. By way of example only, a margin of error of from about 10percent (%) to about 30% may be used for the purposes of validating thepresent data center models. The validated model is considered herein asa “benchmarked” model in that it now can be used to optimize the datacenter. Namely, in step 310, once the model is validated it can be usedfor simulating “what if” scenarios and design optimizations for the datacenter.

For example, the Laplacian model can be used to predict an impact ofchanges in the data center, i.e., racks being moved and/or equipmentbeing edited. In this case, one would add the additional equipment,i.e., more IT racks containing servers (or other IT equipment) and/ormore CRAC units, with the appropriate boundaries. Name plate data (ordiscounted name plate data) might be used to specify the boundaries.Then, Equation 1 and either Equation 2 or Equation 3 are solved topredict the complete temperature field. In contrast, conventional NS-CFDmodels use name plate data for only a small portion of a data center.Further, the present Laplacian model will also have significantly fastercalculation times, as compared to NS-CFD-based models.

The Laplacian model predictions are assumed to be very accurate becausea comparatively larger portion of the data center is measured in detailusing MMT V1.0 (as compared to conventional data center analysistechniques). A benchmarked model based on the MMT V1.0 measurements thusrepresents the data center very accurately. If an error correction arraywas constructed in the initial Laplacian model, then the errorcorrection array can be applied to the predictions in the changed datacenter environment.

While the initial boundaries for the Laplacian model are obtained by MMTV1.0 measurements, other forms of data gathering are also possible. Forexample, even NS-CFD results can be used to define the boundaries in theLaplacian model (which can then be solved for the remaining temperaturefield).

The present techniques provide several advantages over conventionalNS-CFD techniques. First, the data center models described herein can,as described in detail above, be coupled directly to the MMT V1.0measurements. By comparison, standard NS-CFD models require input datathat is generally very difficult to accurately obtain for real-life datacenters (input powers etc.). Namely, in many instances, with standardNS-CFD models the input data is either incomplete or inaccurate and thusthe resulting model does not yield predictions that are commensuratewith measured values.

Second, the present model can be used to obtain linear response matrixesfor several important parameters within the data center (e.g.,perforated tile air flows, CRAC air flows, rack (inlet/outlet air flowsand power levels), which can be exploited for data center optimizationand/or rapid computation. For example, the present models can be used tocalculate temperature fields for each server in a data center. Thesuperposition of these temperature fields weighted with individual powerlevels in each server gives temperature distributions within the datacenter, as follows:

A·P ^(server) =T ^(server),  (4)

wherein A represents a matrix relating to a power for each server P_(i)^(server) with corresponding (inlet) temperatures T_(i) ^(server). T^(server) is a temperature vector of these (inlet) temperatures (T^(server)=[T_(i) ^(server)] for i=1, . . . , n=number of servers), whileP ^(server) is a power vector for each server inlet temperature (P^(server)=[P_(i) ^(server)] for i=1, . . . , n=number of servers).

FIG. 4 is a diagram illustrating air temperature contours and flowprofiles resulting from use of a temperature and air flow model on adata center 400. Namely, data center 400 contains n=11 rows of servers402, each server 402 has a power level P_(i) ^(server). By usingLaplacian modeling a temperature field and corresponding inlettemperatures can be calculated for each server power level.

Any linear superposition of the 11 temperature fields shown in FIG. 4yields a valid new solution for any set of power levels in the datacenter. Specifically, Equation 4, above, can be used to rapidly predicttemperatures for any combination of power levels. The A matrix isdefined by these 11 solutions. For example, the matrix can be obtainedby:

$\underset{\underset{\_}{\_}}{A} = {\begin{pmatrix}{T_{1}^{server}/P_{1}^{server}} & \ldots & {T_{1}^{server}/P_{1}^{server}} \\\vdots & \vdots & \; \\{T_{1}^{server}/P_{1}^{server}} & \ldots & {T_{1}^{server}/P_{1}^{server}}\end{pmatrix}.}$

By applying simple linear and non-linear least square fitting an optimumpower distribution (i.e., to attain an optimum air inlet temperaturedistribution) for the data center can be obtained. Specifically, anoptimum temperature distribution in the data center might be specified,and then Equation 4 can be used to obtain the optimum powerdistribution. See, for example, U.S. Patent Application No. 2007/0098037filed by Hamann et al., entitled “Techniques for Distributing Power inElectronic Circuits and Computer Systems,” the contents of which areincorporated by reference herein. The same concept can be used to obtainthe optimum perforated tile layout in the data center. For example, thepresent model can be used to calculate air flow fields for eachperforated tile in the data center. The superposition of these air flowfields weighted with a supplied air flow from each perforated tile(v_(p)) can yield temperatures within the data center as follows:

F v _(p)=T,  (5)

wherein F represents a linear response matrix between perforated tileair flow vector v_(p) and temperature T. By solving Equation 5 for v_(p)with an optimum temperature distribution T (e.g., using linear orno-linear least square fits) an optimum perforated tile distribution(i.e., how much air flow needs to supplied and to where) for the datacenter can be obtained.

By way of example, Equation 1, above, can be applied to the air flowfield of a data center (e.g., data center 500 of FIG. 5, describedbelow), as follows:

$\begin{matrix}{{{{Perforated}\mspace{14mu} {tile}\mspace{14mu} ({source})\text{:}\mspace{14mu} \frac{\partial\varphi}{\partial y}} = {- v_{y}}},} \\{{{{{CRACs}({sink})}\text{:}\mspace{14mu} \varphi} = 0},} \\{{{{Inlet}\mspace{14mu} {of}\mspace{14mu} {rack}\text{:}\mspace{14mu} \frac{\partial\varphi}{\partial x}} = v_{x}},} \\{{{Outlet}\mspace{14mu} {of}\mspace{14mu} {rack}\text{:}\mspace{14mu} \frac{\partial\varphi}{\partial x}} = {- {v_{x}.}}}\end{matrix}$

FIG. 5 is a diagram illustrating air flow potential and field inexemplary data center 500. As shown in FIG. 5, data center 500 includesfour racks (labeled “RACK #1” through “RACK #4”) and two CRACs (labeled“CRAC #1” and “CRAC #2”). Between CRAC #1 and RACK #1 is a hot aisle(labeled “hot aisle #1”), between RACK #1 and RACK #2 is a cold aisle(labeled “cold aisle #1”), between RACK #2 and RACK #3 is another hotaisle (labeled “hot aisle #2”), between RACK #3 and RACK #4 is anothercold aisle (labeled “cold aisle #2”) and between RACK #4 and CRAC #2 isyet another hot aisle (labeled “hot aisle #3”). In hot aisle #1, theoutlet air flow of RACK #1 is designated at three levels of the rack,high, medium and low, i.e., v_(x) _(—) out_R1_h, v_(x) _(—) out_R1_m andv_(x) _(—) out_R1_(—)1, respectively. In cold aisle #1, the inlet airflows of RACK #1 and RACK #2 are designated at three levels of theracks, high, medium and low, i.e., v_(x) _(—) in_R1_h, v_(x) _(—)in_R1_m and v_(x) _(—) in_R1_(—)1, for RACK #1, and v_(x) _(—) in_R2_h,v_(x) _(—) in_R2_m and v_(x) _(—) in_R2_(—)1, for RACK #2, respectively.Hot and cold air intermixing can occur in cold aisle #2, as shown.

By way of example, Equation 2, above, can be applied to the temperaturefield of a data center (e.g., data center 600 of FIG. 6). FIG. 6 is adiagram illustrating temperature field in exemplary data center 600. Asshown in FIG. 6, data center 600 includes four racks (labeled “RACK #1”through “RACK #4”) and two CRACs (labeled “CRAC #1” and “CRAC #2”).Between CRAC #1 and RACK #1 is a hot aisle (labeled “hot aisle #1”),between RACK #1 and RACK #2 is a cold aisle (labeled “cold aisle #1”),between RACK #2 and RACK #3 is another hot aisle (labeled “hot aisle#2”), between RACK #3 and RACK #4 is another cold aisle (labeled “coldaisle #2”) and between RACK #4 and CRAC #2 is yet another hot aisle(labeled “hot aisle #3”). In hot aisle #1, the outlet air flow of RACK#1 is designated at three levels of the rack, high, medium and low,i.e., v_(x) _(—) out_R1_h, v_(x) _(—) out_R1_m and v_(x) _(—)out_R1_(—)1, respectively. In cold aisle #1, the inlet air flows of RACK#1 and RACK #2 are designated at three levels of the racks, high, mediumand low, i.e., v_(x) _(—) in_R1_h, v_(x) _(—) in_R1_m and v_(x) _(—)in_R1_(—)1, for RACK #1, and v_(x) _(—) in_R2_h, v_(x) _(—) in_R2_m andv_(x) _(—) in_R2_(—)1, for RACK #2, respectively. Hot and cold airintermixing can occur in cold aisle #2, as shown.

The boundaries used in this example are 200 Watts (W) per server (i.e.,for RACK #1, RACK #2 and RACK #3 there are three servers per rack, sothe rack heat load is 600 W each; for RACK #4 there are two servers perrack so the rack heat load is 400 W (for a total heat load of 2,200 Wfor all of the racks)) and a temperature discharged by the perforatedtiles of 12 degrees Celsius (° C.). The greatest server inlettemperature is 25° C. Total power is 2,200 W and total input flow is1,200 cubic feet per minute (cfm).

As described above, a data center can have a sub-floor plenum. By way ofexample, Equations 1 and 2, above, can accommodate sub-floor plenummodeling and other effects in a data center (e.g., data center 700 ofFIG. 7, described below), as follows:

$\begin{matrix}{{{{Inlet}\mspace{14mu} {of}\mspace{14mu} {CRACs}\text{:}\mspace{14mu} \frac{\partial\varphi}{\partial x}} = v_{x}},} \\{{{{Outlet}\mspace{14mu} {of}\mspace{14mu} {CRACs}\text{:}\mspace{14mu} \frac{\partial\varphi}{\partial x}} = {- v_{x}}},} \\{{{\rho \; c_{p}\underset{\_}{v}\mspace{14mu} {{grad}(T)}} + {{div}\left( {{kgrad}(T)} \right)} + h_{Racks} - h_{CRACs}} = 0}\end{matrix}$

FIG. 7 is a diagram illustrating exemplary data center 700 havingsub-floor plenum 702. As shown in FIG. 7, data center 700 includes fourracks (labeled “RACK #1” through “RACK #4”) and two CRACs (labeled “CRAC#1” and “CRAC #2”). Between CRAC #1 and RACK #1 is a hot aisle (labeled“hot aisle #1”), between RACK #1 and RACK #2 is a cold aisle (labeled“cold aisle #1”), between RACK #2 and RACK #3 is another hot aisle(labeled “hot aisle #2”), between RACK #3 and RACK #4 is another coldaisle (labeled “cold aisle #2”) and between RACK #4 and CRAC #2 is yetanother hot aisle (labeled “hot aisle #3”). In hot aisle #1, the outletair flow of RACK #1 is designated at three levels of the rack, high,medium and low, i.e., v_(x) _(—) out_R1_h, v_(x) _(—) out_R1_m and v_(x)_(—) out_R1_(—)1, respectively. In cold aisle #1, the inlet air flows ofRACK #1 and RACK #2 are designated at three levels of the racks, high,medium and low, i.e., v_(x) _(—in) _R1_h, v_(x) _(—) in_R1_m and v_(x)_(—) in_R1_(—)1 for RACK #1, and v_(x) _(—) in_R2_h, v_(x) _(—) in_R2_mand v_(x) _(—) in_R2_(—)1, for RACK #2, respectively. Sub-floor plenum702 is present beneath the racks/CRACs.

Turning now to FIG. 8, a block diagram is shown of an apparatus 800 formodeling a data center, in accordance with one embodiment of the presentinvention. It should be understood that apparatus 800 represents oneembodiment for implementing methodology 200 of FIG. 2 and/or methodology300 of FIG. 3, both described above.

Apparatus 800 comprises a computer system 810 and removable media 850.Computer system 810 comprises a processor 820, a network interface 825,a memory 830, a media interface 835 and an optional display 840. Networkinterface 825 allows computer system 810 to connect to a network, whilemedia interface 835 allows computer system 810 to interact with media,such as a hard drive or removable media 850.

As is known in the art, the methods and apparatus discussed herein maybe distributed as an article of manufacture that itself comprises amachine-readable medium containing one or more programs which whenexecuted implement embodiments of the present invention. For instance,the machine-readable medium may contain a program configured to obtainspatially dense three-dimensional thermal distribution and air flowmeasurements made in the data center using a mobile off-line surveyingsystem; create a temperature and air flow model for the data centerusing the spatially dense three-dimensional thermal distribution and airflow measurements; use the temperature and air flow model to makethermal distribution and air flow predictions of the data center; andcompare the thermal distribution and air flow predictions with thethermal distribution and air flow measurements made using the mobileoff-line surveying system to produce a validated model for the datacenter. The program may, for example, also be configured to determinewhether a margin of error between the thermal distribution and air flowpredictions and the thermal distribution and air flow measurements isacceptable; make one or more changes to the temperature and air flowmodel if the margin of error between the thermal distribution and airflow predictions and the thermal distribution and air flow measurementsis unacceptable; and repeat the use, compare and determine steps untilthe margin of error between the thermal distribution and air flowpredictions and the thermal distribution and air flow measurements isacceptable.

The machine-readable medium may be a recordable medium (e.g., floppydisks, hard drive, optical disks such as removable media 850, or memorycards) or may be a transmission medium (e.g., a network comprisingfiber-optics, the world-wide web, cables, or a wireless channel usingtime-division multiple access, code-division multiple access, or otherradio-frequency channel). Any medium known or developed that can storeinformation suitable for use with a computer system may be used.

Processor 820 can be configured to implement the methods, steps, andfunctions disclosed herein. The memory 830 could be distributed or localand the processor 820 could be distributed or singular. The memory 830could be implemented as an electrical, magnetic or optical memory, orany combination of these or other types of storage devices. Moreover,the term “memory” should be construed broadly enough to encompass anyinformation able to be read from, or written to, an address in theaddressable space accessed by processor 820. With this definition,information on a network, accessible through network interface 825, isstill within memory 830 because the processor 820 can retrieve theinformation from the network. It should be noted that each distributedprocessor that makes up processor 820 generally contains its ownaddressable memory space. It should also be noted that some or all ofcomputer system 810 can be incorporated into an application-specific orgeneral-use integrated circuit.

Optional video display 840 is any type of video display suitable forinteracting with a human user of apparatus 800. Generally, video display840 is a computer monitor or other similar video display.

As highlighted above, the CFD methodology using the NS equations orpotential flow techniques and equations can be used to generate a datacenter model. FIG. 9 is a diagram illustrating exemplary methodology 900for creating a temperature flow model, such as a NS-CFD model or apotential flow model, using MMT V1.0 measurements. Data center modelingtools using CFD methods having a wide library of data center buildcomponents (such as perforated tiles, CRAC units and IT rack equipment)are commercially available and can be used in conjunction withmethodology 900. As highlighted above, in the present data center modelskey inputs are perforated tile air flow rates and temperatures, rackheat loads and air flow rates, CRAC air flow rates, as well as severalmiscellaneous details, such as tile perimeter leakage flow rates,location of cable openings and direction of CRAC unit flow. Methodology900 is directed to the rapid generation of the inputs required toconstruct the model.

In step 902, a variety of different data collection activities areconducted, including, but not limited to, a visual inspection of theracks, three-dimensional temperature mapping using MMT V1.0, perforatedtile air flow measurements, perforated tile temperature measurements,layout definition (layout information of the facility (data center) isgathered), rack inlet and outlet temperature measurements, exit velocityto rack (i.e., velocity at rear (back) of the rack or at an exhaustregion of a front-to-back server rack) measurements and total inputpower to the data center measurements. In step 904, the NS-CFD modelinputs for the racks are defined, i.e., factors and calculations thatwill help estimate rack parameters, such as rack power levels and flow.According to an exemplary embodiment, weighting factors are assigned torack power levels (using rack inlet and outlet temperatures and exitvelocities), flow per sever node for the racks is estimated, rack powersare calculated using total power and weighting factors and rack flowsare calculated using exit velocity and server information. While thetotal rack power is often known via measured data from the PDUs, thepower of each individual rack needs to be determined via name plateinformation, individual rack configurations and an estimation ofcomputational intensity of operation of the IT equipment.

In step 906, additional parameters are estimated, such as air flowparameters related to the CRACs, the perforated tiles and data centerair leakage flow via miscellaneous openings. A mass average of tiletemperatures and CRAC, lighting and IT equipment power estimates canalso be made.

In step 908, all of the necessary input information is entered into theNS-CFD model (or potential flow or Laplacian model). According to anexemplary embodiment, a model shell is first built based on the datacenter layout information. Next, the tile air flow data and an averageof the tile temperature data are entered into the model. The air leakageflow estimates for the perforated tiles and miscellaneous openings areused in the model. Lighting loads are added to the model as a powerinput.

In step 910, the NS-CFD model (or potential flow or Laplacian model) issolved and three-dimensional temperature data obtained from the model isorganized in a manner that facilitates comparison to the correspondingdata collected using MMT V1.0. This step also includes a calculation oferror via simple subtraction, or by means of more sophisticatedstatistical metrics, to quantify the accuracy of the model. According toan exemplary embodiment, the temperature data from the model and thetemperature data from MMT V1.0 are compared point by point inthree-dimensional space, and global and local error metrics andstatistics are generated.

In step 912, a decision is made regarding the accuracy of the modelresults. A negative answer results in changes to the rack model inputsfrom step 904, while a positive answer allows for use of the model, instep 914, for simulating “what if” scenarios and design optimizations.As highlighted above, an error range of from about 10% to about 30% isreasonable for data centers.

FIGS. 10A-C are plan view temperature contour plots for a single heightof 5.5 feet illustrating a comparison of MMT V1.0 data and NS-CFD (orpotential flow or Laplacian) model data with an unacceptable amount oferror present. Namely, FIG. 10A is a plan view temperature contour plotof the NS-CFD (or potential flow or Laplacian) model results, FIG. 10Bis a plan view temperature contour plot of the experimental (MMT V1.0)data and FIG. 10C is a plan view temperature error contour plot of thedifference between the model and experimental data (model—experimentaldata). As seen in FIGS. 10A and 10B, both the model and experimentaldata show a hot spot region to exist in top-central and top-rightregions of the plan view. While there appears to be much more pronouncedhot and cold regions in the model data, the experimental data shows more“mixed” temperature values. These results can be interpreted as that themodel inhibits the mixing between hot and cold air streams as comparedto what is observed in reality. The data as illustrated in FIG. 10B alsoshows less pronounced hot and cold regions when looking at the plan viewin horizontal section from the top to the bottom, i.e., by aisle. Onexamining FIG. 10A in the same manner, one observes a much morepronounced spatial non-uniformity in the temperature contours. FIG. 10Cshows the temperature error contours yielding high values of modeltemperature compared to the data in the top right hand corner of theplan view. The difference in size of the white space between the modeland experimental data contour plots is due to MMT V1.0 needing a bit ofclearance between itself and the racks, and MMT V1.0 can also not reachevery portion of the data center. The errors in this case are consideredto be unacceptably high and the NS-CFD (or potential flow or Laplacian)model is “asked” to make changes to its inputs.

FIGS. 11A-C show the corresponding temperature plots and error plotsafter these changes have been made, the model has been run again and theresults have been compared to the experimental (MMT V1.0) data. Namely,FIG. 11A is a plan view temperature contour plot of the updated NS-CFD(or potential flow or Laplacian) model results, FIG. 11B is a plan viewtemperature contour plot of the experimental (MMT V1.0) data and FIG.10C is a plan view temperature error contour plot of the differencebetween the model and experimental data (model—experimental data). Inthis case, the errors are lower than an acceptable threshold (anacceptable amount of error is present) and the model can now be use forfurther analyses.

Although illustrative embodiments of the present invention have beendescribed herein, it is to be understood that the invention is notlimited to those precise embodiments, and that various other changes andmodifications may be made by one skilled in the art without departingfrom the scope of the invention.

1. A method for modeling a data center, comprising the steps of:obtaining spatially dense three-dimensional thermal distribution and airflow measurements made in the data center using a mobile off-linesurveying system; creating a temperature and air flow model for the datacenter using the spatially dense three-dimensional thermal distributionand air flow measurements; using the temperature and air flow model tomake thermal distribution and air flow predictions of the data center;and comparing the thermal distribution and air flow predictions with thethermal distribution and air flow measurements made using the mobileoff-line surveying system to produce a validated model for the datacenter.
 2. The method of claim 1, wherein the mobile off-line surveyingsystem is mobile measurement technology V1.0.
 3. The method of claim 1,wherein the temperature and air flow model created is aNavier-Stokes-based temperature and air flow model.
 4. The method ofclaim 1, wherein the temperature and air flow model created is aLaplacian model having separate temperature and air flow fields.
 5. Themethod of claim 4, wherein the temperature field is represented byρc_(p) vgrad(T)+div(k grad(T))+h=0, wherein ρ is air density, c_(p) isspecific heat, v is velocity vector, T is absolute temperature, h ispower dissipation/heat removal and k is thermal conductivity.
 6. Themethod of claim 4, wherein the temperature field is represented by${{{\frac{\partial}{\partial x}\left\lbrack {{\alpha \frac{\partial T}{\partial x}} - {v_{x}T}} \right\rbrack} + {\frac{\partial}{\partial y}\left\lbrack {{\alpha \frac{\partial T}{\partial y}} - {v_{y}T}} \right\rbrack} + {\frac{\partial}{\partial z}\left\lbrack {{\alpha \frac{\partial T}{\partial z}} - {v_{z}T}} \right\rbrack}} = 0},$wherein T is absolute temperature, v_(x) is air flow velocity in xdirection, v_(y) is air flow velocity in y direction and v_(z) is airflow velocity in z direction.
 7. The method of claim 4, wherein the airflow field is represented by ∇²φ=0, wherein φ is flow potential.
 8. Themethod of claim 7, further comprising the step of: setting an actualrelative value for the flow potential somewhere in the data center. 9.The method of claim 1, further comprising the steps of: determiningwhether a margin of error between the thermal distribution and air flowpredictions and the thermal distribution and air flow measurements isacceptable; making one or more changes to the temperature and air flowmodel if the margin of error between the thermal distribution and airflow predictions and the thermal distribution and air flow measurementsis unacceptable; and repeating the using, comparing and determiningsteps until the margin of error between the thermal distribution and airflow predictions and the thermal distribution and air flow measurementsis acceptable.
 10. The method of claim 1, further comprising the stepof: calculating an error correction array if the margin of error betweenthe thermal distribution and air flow predictions and the thermaldistribution and air flow measurements is unacceptable.
 11. The methodof claim 1, further comprising the step of: using the temperature andair flow model to optimize the data center.
 12. An apparatus formodeling a data center, the apparatus comprising: a memory; and at leastone processor, coupled to the memory, operative to: obtain spatiallydense three-dimensional thermal distribution and air flow measurementsmade in the data center using a mobile off-line surveying system; createa temperature and air flow model for the data center using the spatiallydense three-dimensional thermal distribution and air flow measurements;use the temperature and air flow model to make thermal distribution andair flow predictions of the data center; and compare the thermaldistribution and air flow predictions with the thermal distribution andair flow measurements made using the mobile off-line surveying system toproduce a validated model for the data center.
 13. The apparatus ofclaim 12, wherein the temperature and air flow model created is aLaplacian model having separate temperature and air flow fields.
 14. Theapparatus of claim 13, wherein the temperature field is represented byρc_(p) v grad (T)+div(k grad(T))+h=0, wherein ρ is air density, c_(p) isspecific heat, v is velocity vector, T is absolute temperature, h ispower dissipation/heat removal and k is thermal conductivity.
 15. Theapparatus of claim 13, wherein the temperature field is represented by${{{\frac{\partial}{\partial x}\left\lbrack {{\alpha \frac{\partial T}{\partial x}} - {v_{x}T}} \right\rbrack} + {\frac{\partial}{\partial y}\left\lbrack {{\alpha \frac{\partial T}{\partial y}} - {v_{y}T}} \right\rbrack} + {\frac{\partial}{\partial z}\left\lbrack {{\alpha \frac{\partial T}{\partial z}} - {v_{z}T}} \right\rbrack}} = 0},$wherein T is absolute temperature, v_(x) is air flow velocity in xdirection, v_(y) is air flow velocity in y direction and v_(z) is airflow velocity in z direction.
 16. The apparatus of claim 13, wherein theair flow field is represented by ∇²φ=0, wherein φ is flow potential. 17.The apparatus of claim 12, wherein the at least one processor is furtheroperative to: determine whether a margin of error between the thermaldistribution and air flow predictions and the thermal distribution andair flow measurements is acceptable; make one or more changes to thetemperature and air flow model if the margin of error between thethermal distribution and air flow predictions and the thermaldistribution and air flow measurements is unacceptable; and repeat theuse, compare and determine steps until the margin of error between thethermal distribution and air flow predictions and the thermaldistribution and air flow measurements is acceptable.
 18. An article ofmanufacture for modeling a data center, comprising a machine-readablemedium containing one or more programs which when executed implement thesteps of: obtaining spatially dense three-dimensional thermaldistribution and air flow measurements made in the data center using amobile off-line surveying system; and creating a temperature and airflow model for the data center using the spatially densethree-dimensional thermal distribution and air flow measurements; usingthe temperature and air flow model to make thermal distribution and airflow predictions of the data center; and comparing the thermaldistribution and air flow predictions with the thermal distribution andair flow measurements made using the mobile off-line surveying system toproduce a validated model for the data center.
 19. The article ofmanufacture of claim 18, wherein the temperature and air flow modelcreated is a Laplacian model having separate temperature and air flowfields.
 20. The article of manufacture of claim 19, wherein thetemperature field is represented by ρc_(p) v grad(T)+div(k grad(T))+h=0,wherein ρ is air density, c_(p) is specific heat, v is velocity vector,T is absolute temperature, h is power dissipation/heat removal and k isthermal conductivity.
 21. The article of manufacture of claim 19,wherein the temperature field is represented by${{{\frac{\partial}{\partial x}\left\lbrack {{\alpha \frac{\partial T}{\partial x}} - {v_{x}T}} \right\rbrack} + {\frac{\partial}{\partial y}\left\lbrack {{\alpha \frac{\partial T}{\partial y}} - {v_{y}T}} \right\rbrack} + {\frac{\partial}{\partial z}\left\lbrack {{\alpha \frac{\partial T}{\partial z}} - {v_{z}T}} \right\rbrack}} = 0},$wherein T is absolute temperature, v_(x) is air flow velocity in xdirection, v_(y) is air flow velocity in y direction and v_(z) is airflow velocity in z direction.
 22. The article of manufacture of claim19, wherein the air flow field is represented by ∇²φ=0, wherein φ isflow potential.
 23. The article of manufacture of claim 18, wherein theone or more programs which when executed further implement the steps of:determining whether a margin of error between the thermal distributionand air flow predictions and the thermal distribution and air flowmeasurements is acceptable; making one or more changes to thetemperature and air flow model if the margin of error between thethermal distribution and air flow predictions and the thermaldistribution and air flow measurements is unacceptable; and repeatingthe using, comparing and determining steps until the margin of errorbetween the thermal distribution and air flow predictions and thethermal distribution and air flow measurements is acceptable.